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AN  OPTIMUM  IDENTIFYING  DEVICE  FOR  SYSTEMATIC  CODES 
AND  CERTAIN  TYPES  OF  CHANNELS 

I.  I.  Grushko 

o 

A  method  of  constructing  an  optimum  Identifying 
device  for  systematic  correcting  codes  Is  considered. 

In  this  case  It  Is  assumed  that  the  channel  Is 
given  at  the  output  of  the  threshold  device  by  the 
stochastic  matrix  of  the  conversion  of  the  elemen¬ 
tary  signals. 

A  number  of  authors  have  suggested  transmitting  messages  through 
discrete  communication  channels  with  the  aid  of  uniform  n-valued 
correcting  codes  with  the  base  y  and  m  Information  positions..  In 
this  case  the  following  sequence  of  symbols  enters  the  transmitter 
Input  (3ee  figure) 

A’*  —  *„<*>)  (Aj  =  1,2,...,  7m)  (l) 

The  transmitter  converts  into  a  complex  signal  Y^,  which  Is 
a  set  of  elementary  signals  (not  necessarily  time-separated) 

Y*  —  jrjW y„,w*  ( ^ ) 

fv]  flf  ] 

where  the  elementary  signal  yj  '  -corresponds  to  the  symbol  x'  ' .  It 
is  assumed  that  the  elementary  signals  are  distinguished  by  one  param- 
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eter  (amplitude,  phase,  etc.)* 
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As  a  result  of  the  action  of  the  channel  noise  on  the  signal  (2), 
the  following  random  signal  will  be  obtained  at  the  receiver  input 

M  H  (5) 

(lr) 

where  hj  is  a  distorted  elementary  signal  yj  ' .  Prom  the  random 
signal  (5)  the  following  signal  will  be  formed  at  the  receiver  output 

Y  =  (y« . in . y»)  ^ 

Here  y^  is  a  random  value  of  the  parameter  ^  of  the  signal  hj . 

The  receiver  is  followed  by  a  threshold  device,  in  which  the  assumed 
value  of  the  parameter  £  is  identified  with  one  of  the  possible  values. 
At  the  output  of  the  threshold  device  the  result  of  the  action  of  the 
noise  on  the  elementary  signals  is  represented  in  the  form  of  the 
stochastic  matrix  of  the  transformation  of  the  symbols 

U»*ll  (5) 

where  is  the  probability  that  in  transmitting  the  i-th  symbol 
it  will  be  received  as  the  j-th  at  the  output  of  the  threshold  device. 
Henceforth  we  shall  regard  this  matrix  as  given.  We  shall  also  assume 
that  the  nose  distorts  the  elementary  signals  independently..  At  .the 
output  of  the  threshold  device  we  obtain  a  certain  code  combination 

Z  =  (*• 2») 


which,  generally  speaking,  does  not  coincide  with  any  one  of  the 
possibly  transmitted  ones.  Therefore,  the  threshold  device  is  followed 
by  an  identifying  device,  which  identifies  the  obtained  code  combina- 
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tion  Z  with  one  of  the  possibly  transmitted  ones  X^  according  to 
some  indicator  or  other.  The  meet  widespread  indicator  used  in. the 
existing  literature  is  the  minimum  distance  (in  the  sense  of  Hemming) 

4' 

between  the  code  combinations  Z  and  X^ .  Obviously,  in  choosing  an 
appropriate  indicator  it  is  necessary  to  proceed  from  a  certain 
criterion  and  type  of  stochastic  matrix  (5). 

Let  us  take  as  the  basic  criterion  the  maximum  probability  of 
correct  reception  of  the  code  combination,  and  let  us  consider  how 
to  construct  an  Identifying  device  in  the  best  possible  way  for  the 

chosen  code  (i.e.,  a  certain  fixed  set  of  7m  code  combinations  out 

*  v 

of  all  the  possible  y  )  and  for  the  given  channel  (i.e.,  the  given 
matrix  ||  p±J  ||  ). 

Let  us  impose  one  more  restriction:  we  shall  assume  the  given 
code  to  be  systematic.  This  means  that  the  code  combinations  can 
be  written  in  the  form 

A'*  =  (x,<*> . +  ,<*> . *„<*>)  (k  =  i  ,2 . r") 

*,<«  . i-m  +  1,  m  +  2,...,n  (l') 

where  .  a%/)  are  certain  linear  functions  which  determine  the 

correcting  positions  of  the  code  combination  in  terms  of  Information 

positions.  Moreover,  the  functions  . *w)-  have  the  following 

conditions  imposed  on  them: 

1)  Each  separate  correcting  position  is  formed  according  to 
completely  determined  rules,  which  are  always  the  same  for  a  given 
code  combination; 

2)  The  numbers  of  the  positions  being  verified  do  not  depend 
on  a  set  of  information  symbols,  i.e.,  on  which  of  the  numbers  0,  1, 
...,  7—  1  occupy  the  Information  positions  of  the  given  code  combina 
tion. 
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The  restriction  imposed  has  no  essential  significance,  since  it 
follows  from  the  literature  [1,  2]  that  systematic  codes  behave  with 
respect  to  noise  immunity  and  -speed  of  transmission  of  information 
in  approximately  the  same  way  as  nonsystematic  codes.  On  the  other 
hand,  this  restriction  has  a  number  of  advantages: 

1)  For  systematic  codes  there  exists  a  well  developed  apparatus 
of  elementary  group  theory; 

2)  All  the  code  combinations  behave  identically  during  transmis¬ 
sion; 

5)  The  coding  is  accomplished  fairly  simply; 

4)  in  certain  cases  there  are  no  better  codes; 

5)  For  certain  types  of  channels  it  is  possible  to  construct 
an  optimum  identifying  device  that  is  simple  and  lends 
itself  to  practical  realization. 

Let  us-  consider  how  to  construct  an  identifying  device  for  the 
chosen  code  and  the  given  channel  in  the  best  possible  way  with  respect 
to  the  assumed  criterion  of  maximum  probability  of  correct  reception 
of  a  code  combination. 

Let  the  channel  be  described  by  the  stochastic  matrix  ||  p^j  ||  of 
the  transformation  of  the  elementary  signals.  Since,  by  assumption, 
the  channel  noise  distorts  the  individual  positions  of  the  code 
combination  independently,  the  probability  of  a  conversion  of  the 
code  combination  X  =  (x^,  ...,  xn)  into  the  code  combination  Y  -  (y^, 

. , . ,  y  )  at  the  output  of  the  threshold  will  be  written  in  the  form 

(6) 

p(x~y)-n*4w 

Let  us  assume  that 

^l-WT 

where  f(X,  Y)  is  a  function  of  X  and  Y  satisfying  the  condition 


/(*.  y>-/(x  +  z.  y  +  z) 


(7) 


for  any  X,  Y,  and  Z. 

Let  a  systematic  code  A,  determined  by  formula  (l),  be  given 
(note  that  among  the  elements  of  the  code  there  always  exists  a  zero 
element,  i.e.,  a  code  combination  Xq  *=  (0,  0,  . ..,  0)  .  Let  us  choose 
a  certain  element  Y^,  which  does  not  belong  to  set  A,  and  construct 
a  set  A^  of  all  code  code  combinations  of  the  type 

Yl  +  Xl 

where  X^(i  =  1,  2,  7m)  runs  through  all  the  elements  of  code  A. 

Then  let  us  choose  an  element  Yg,  which  belongs  neither  to  A  nor 
A^,  and  construct  a  set  Ag  of  all  code  combinations  of  the  type 

Y2  +X±,  XieA 

And  so  on  until  we  exhaust  the  set  of  all  possible  n-valued  code  combi¬ 
nations  with  elements  assuming  one  of  the  values:  0,  1  ...,  7  -  1. 

It  can  be  shown  [jl  that  the  sets  A,  A^,  Ag  ...  thus  constructed  have 
an  equal  number  of  elements  and  do  not  intersect,  i.e.,  there  does 
not  exist  a  code  combination  such  that  it  would  belong  simultaneously 
to  more  than  one  of  the  sets  A,  A^,  Ag,  . . .  Let  us  write  out  the  sets 


A Ag,  ...  in  rows  in  the 

form  of 

a  table: 

0  Cod©  A 

x.  xt 

A*. 

s*t 

v.  v;+*, 

yi  +  ^s  .  .  . 

y.+r^ 

y.  y.+*« 

V.+  A,  .  . . 

y»+^ 

•4*“(Y*+AV,  Xt£A) 

y*  Vs  +  xx 

Y*  +  X,  .  .  . 

y.+AV 

where  p.  =  7m  and  v  =  yn~m. 


It  can  be  readily  demonstrated  that  any  systematic*  code  (1) 
Is  a  subgroup  in  the  additive  group  of  all  n-dlmenslonal  sequences 
over  the  ring  of  residue  classes  modulo  7  with  respect  to  the 
operation  of  coordinate-wise  addition. 
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We  shall  call  the  table  normal  if  j 

» 

/ (XttYk)  < ](X„  Y k  +  Xt)  (8)  j 

Prom  elementary  group  theory  [j]  It  follows  that  any  of  the  ; 

elements  of  the  set  Aj  may  be  chosen  as  the  generating  element  of 
Aj .  Therefore,  In  constructing  a  normal  table,  it  Is  sufficient  to 
choose  from  each  row  of  the  table  an  element  such  that  condition 
(8)  is , satisfied,  after  which  an  appropriate  permutation  is  made. 

In  accordance  with  the  literature  [4],  let  us  use  the  following 
scheme  (Slepian's  detector)  for  the  identifying  device:  '  if  an 
element  Z,  obtained  at  the  output  of  the  threshold  device,  is  located 
.  in  the  i-th  column  of  the  normal  table,  the  detector  gives  the  code 
combination  at  the  receiving  end. 

The  following  theorem  is  valid: 

Slepian's  detector  is  a  maximally  reliable  scheme  of  identification 

for  a  channel  satisfying  condition  (j),  i.e.,  for  the  given  code  A 

'  ,  I 

it  gives  the  greatest  probability  of  correct  reception  of  a  code 
combination. 

Actually,  the  code  combination  being  used  occupies  a  certain 
definite  position  in  the  normal  table,  i.e.,  there  exists  a  pair  of 
subscripts  k  and  such  that 

Z=Yk+Xj 

Talcing  into  account  that  the  following  formula  Is  always  valid 
for  any  code  combination  T 

XQ+  T  =  T  1 

V  l 

We  obtain  from  condition  (7)  j 

/  (X9t  yk)  a*  /  (x§  Xj,  Yk  +  Xj) « /  (x^Yit  -j-  Xj)  I 

Yk  +  -X») •—  /  +  Xjt  Yk  +  Xi  -f  Xj)  =  f  {X],  Yk  + Xi) 

\ 

fop  any  J  ^  Z  *  1;  2^  •  •  •  p  [i*;  1c  ®  I *  2|  •  •  •  k  v  *  In  this  case  the  j 
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element  »  X^  +  Xj  runs  through  the  entire  cofle  A  for  fixed  1,  and 
any  j  *  lf  2,  . ..,  |t.  Substituting  the  result  obtained  In  (8),  we. 
obtain 


f(Xi,  X„  +  X,)  </(Xj,  V*  +  X,) 

'  > 


for  any  J ,  l 
condition 


1,  2,  .. .,  nj  k  =  1,  2,  v.  By  virtue  of  the 


p<x-^=Tuhn 


This  means  that  Sleplan's  detector  ensures  the  greatest  probability 
of  correct  reception  of  a  code  combination  for  a  given  code  and  channel. 

Let  us  consider  certain  examples. 

1.  In  most  of  the  literature  on  coding  (Sleplan's,  in  particular 
[4] )  a  so-called  binary  symmetrical  channel  is  analyzed.  Let  us 
consider  a  symmetrical  channel  with  any  number  of  elementary  signals 
7.  This  channel  is  described  by  the  stochastic  matrix  ||p^j||  l 


PH  “  Pi+t.  i  =  Pi.  i+ 1  “  p  (i+j)  '  ' 

for  all  1,  J  =  0,  1,  . . . ,  7  -  1. 

By  virtue  of  conditions  (9)  the  value  of  (6)  in  the  given  case 
depends  only  on  the  number  of  positions  in  which  the  code  combination 
X  differs  from  the  code  combination  Y,  i.e.,  the  probability  of  a 
conversion  of  the  code  combination  X  into  the  code  combination  Y  is 
a  function  of  the  Hemming  distance  p(X,  Y)  between  the  code  combinations 
X  and  Y  and  is  equal  to 


P  (X— y)  o  pHX.Y)qr 

(the  Hemming  distance  p(X,  Y)  between  code  combinations  X  and  Y  is 
determined  as  the  number  of  positions  in  whloh  X  and  Y  differ  from 
each  other).  By  virtue  of  a  coordinate-wise  determination  of  the 
operation  of  addition  of  the  code  combinations,  the  Hemming  distance 
p(X,  Y)  satisfies  the  condition 
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P(X,  Y)  =  p(X  +  Z.Y+Z) 


for  any  code  combinations  X,  Y,  and  Z. 

Therefore,  choosing  as  the  function  f (X,  Y)  the  quantity 

we  can  construct  an  optimum  identifying  device.  Since 

In/  (X,  y)  =  f>  (X,  Y)  la  -J-  -  nhiq 

the  inequality  (8),  by  virtue  of  the  continuity  of  a  logarithmic 
function,  leads  to  the  inequality 


P(X„  y*)<p(A\,  y*’+ X,) 


for  all  k  =  1,  2,  ... .,  v  and  i  =  1,  2,  ...,  p..  This  means  that  in 
order  to  construct  a  normal  table  determining  an  optimum  identifying 
device,  the  elements  Yj(j  =  1,  2,  ...,  v)  must  be  chosen  in  such  a 
way  that  within  the  space  of  the. signal  they  are  not  located  beyond 
(in  the  sense  of  Hemming  distance)  all  the  other  elements  of  the  set 
Aj  =  (Yj  +  X1;  X^gA) .  The  result  obtained  agrees  with  Slepianb  [4]  in 
the  case  of  a  binary  symmetrical  channel. 


2.  The  stochastic  matrix  of  the  channel 

Pu  =  g 


'iJ 


has  the  form: 


•  =  «—•<*— j>*  , 

for  all  .1,  j  =  Q  1,  ...,  7-1.  In  this  case  the  probability  of  a 
conversion  of  a  code  combination  X  into  a  code  combination  Y  will 
be  written  in  the  form 


p  {X  —  Y)  =  q*e—WX .  V)  _  gH—piX,  y)  g-tdHX.  Y) 

where  k  is  the  number  of  coinciding  positions  in  the  code  combinations 
X  and  Y,  while  d(X,  Y)  is  an  ordinary  distance  in  a  Euclidean  metric. 

By  virtue  of  a  coordinate-wise  determination  of  the  addition  of 
the  code  combinations,  we  have 
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HX,Y)-HX+Z,Y  +  Z),  -  ?(X,Y)-HX  +  Z.Y+Z) 

Thus,  choosing  as  f(x,  y)  the  function 

/{X,  Y)  -  f**. 

we  can  construct  an  optimum  identifying  device  in  the  way  described 
above.  . 


Received  on  January  31,  1961 
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